Typical computer systems, especially computer systems using graphical user interface (GUI) systems such as Microsoft WINDOWS, are optimized for accepting user input from one or more discrete input devices such as a keyboard for entering text, and a pointing device such as a mouse with one or more buttons for driving the user interface. The ubiquitous keyboard and mouse interface provides for fast creation and modification of documents, spreadsheets, database fields, drawings, photos and the like. However, there is a significant gap in the flexibility provided by the keyboard and mouse interface as compared with the non-computer (i.e., standard) pen and paper. With the standard pen and paper, a user edits a document, writes notes in a margin, and draws pictures and other shapes and the like. In some instances, a user may prefer to use a pen to mark-up a document rather than review the document on-screen because of the ability to freely make notes outside of the confines of the keyboard and mouse interface.
Some computer systems permit a user to write on a screen using, for example, a stylus. For example, the Microsoft READER application permits one to add electronic ink (also referred to herein as “ink”) to a document much the same way that a user would write with a standard pen and paper. Most hand-held computing devices, commonly known as Personal Digital Assistants (PDAs), also permit the user to write on the screen.
A handwriting recognition system may then be used to analyze the electronic ink to recognize characters, for example, Unicode characters. As the user moves the stylus across the screen, the computing device senses the position of the stylus as the user writes and stores the position data. The computing device analyzes the position data and converts it to recognized characters, such as letters or numbers, in a convenient format, such as Unicode format.
In particular, the handwriting recognition system uses algorithms to map handwritten data to characters. For example, the system may store training data for each character that can be recognized. The training data allows the system to map the user's input to characters. As long as the user writes like the training data, the handwritten data is successfully recognized. Conversely, the more dissimilar the user's input and the training data are, the more likely it is that the handwritten data will be misrecognized.
Over the years, handwriting recognizing systems have evolved to more closely simulate the user's normal writing experience with a pen and paper. For example, early handwriting recognizing systems required the user to write each letter in a separate box. Subsequent handwriting recognizing systems moved away from the box and even allowed the user to write in cursive, however, required the user to write on a horizontal line. Even subsequent evolutions allowed the user to write anywhere on the user interface screen but still required the handwriting to be horizontal.
Although handwriting recognition systems have grown in popularity and flexibility for computing devices, they do not accommodate certain common writing habits of the user. For example, existing handwriting recognizing systems require the user to write horizontally. As the user deviates further from writing horizontally, the handwriting recognition system quickly starts mis-recognizing characters. In fact, a slight deviation from the horizontal writing axis may quickly result in a 100% error rate in handwriting recognition. This limits the user's interaction with, for example, a tablet PC where the user may naturally deviate from a horizontal writing path. Also, user interaction is limited where the user seeks to annotate a document on the margins with comments that may be written at an angle.
Another limitation is that existing handwriting recognizing systems cannot recognize whether the user has handwritten multiple angled lines of information. For example, the user when annotating a document may write in multiple angled lines. Existing handwriting recognition systems, however, are not capable of recognizing whether the user is writing on multiple lines where the user is not writing on a horizontal line.
Accordingly, to better mirror the user's experience with writing on a traditional pad of paper, it is therefore desirable to process electronic ink for handwriting recognition in a manner that overcomes one or more of the above problems.